Forecast on Hospitalisation (ARIMA Modelling)

Data Preparation

#For forecasting, we chose the latest data
trend_hosp_hb <- trend_hb_daily %>% 
  filter (hb_name == "Scotland") %>% 
  filter(date >="2021-06-01") %>% 
  filter(!(is.na(hospital_admissions))) %>% 
  select(date, hospital_admissions)

# Convert it into a time series
daily_hosp_hb_zoo <- zoo(trend_hosp_hb$hospital_admissions, 
           order.by=as.Date(trend_hosp_hb$date, format='%m/%d/%Y'))

# Convert it into a time series
daily_hosp_hb_timeseries <-  timeSeries::as.timeSeries(daily_hosp_hb_zoo)

Step 1 : Visualize the time series

original_series<-autoplot(daily_hosp_hb_timeseries, ts.colour = '#5ab4ac')+
  xlab("Month") + 
  ylab("Number of People hospitalised")+
  #scale_x_date(breaks = "1 month", date_labels = "%b - %y" )+
  theme(axis.text.x = element_text(angle = 45, vjust = 1, hjust=1))+
  ggtitle("Trend on Hospitalisation") +
  color_theme()

ggplotly(original_series)

Step 2 : Identification of model : (Finding d:)

Identify whether the time series is stationary / non stationary we can use ADF Augmented Dickey-Fuller test

adf_test_hosp <- adf.test(daily_hosp_hb_timeseries)
adf_test_hosp

    Augmented Dickey-Fuller Test

data:  daily_hosp_hb_timeseries
Dickey-Fuller = -1.0883, Lag order = 4, p-value = 0.9209
alternative hypothesis: stationary

The time series is not stationary since we have a high p-value (p-value must be < 0.05). So we apply difference

first_diff_hosp<- diff(daily_hosp_hb_timeseries)
adf_test1_hosp <- adf.test(na.omit(first_diff_hosp))

Warning in adf.test(na.omit(first_diff_hosp)) :
  p-value smaller than printed p-value

adf_test1_hosp

    Augmented Dickey-Fuller Test

data:  na.omit(first_diff_hosp)
Dickey-Fuller = -7.5072, Lag order = 4, p-value = 0.01
alternative hypothesis: stationary

Create a dataframe to compare

adf_data_hosp <- data.frame(Data = c("Original", "First-Ordered"),
                       Dickey_Fuller = c(adf_test_hosp$statistic, adf_test1_hosp$statistic),
                       p_value = c(adf_test_hosp$p.value,adf_test1_hosp$p.value))
adf_data_hosp

Initially the p-value is high which indicates that the Time Series is not stationary. So we apply difference 1 time. After the first difference, the p-value < significance level (0.05) So we can conclude that the difference data are stationary. So difference (d = 1)

Other method:

ndiffs(daily_hosp_hb_timeseries)

Let’s plot the First Order Difference Series

Order of first difference

p<- autoplot(first_diff_hosp, ts.colour = '#5ab4ac') +
  xlab("Month") + 
  ylab("HOSPITALIZATION")+
 # scale_x_date(breaks = "1 month", date_labels = "%b - %y" )+
  theme(axis.text.x = element_text(angle = 45, vjust = 1, hjust=1))+
  ggtitle("First-Order Difference Series") +
  color_theme()

ggplotly(p)

Step 3: Estimate the parameters (Finding p and q)

For our model ARIMA (p,d,q), we found d = 1, the next step is to get the values of p and q, the order of AR and MA part. Plot ACF and PACF charts to identify q and p respectively.

par(mfrow=c(2,2))
acf_hosp  <- acf1(daily_hosp_hb_timeseries, col=2:7, lwd=4)
pacf_hosp <- acf1(daily_hosp_hb_timeseries,  pacf = TRUE, col=2:7, lwd=4)
acf_hosp  <- acf1(first_diff_hosp, col=2:7, lwd=4)
pacf_hosp <- acf1(first_diff_hosp,  pacf = TRUE, col=2:7, lwd=4)

The ACF and PACF plots of the differenced data show the following patterns:

The ACF is sinusoidal and there is a significant spike at lag 3 in the PACF So the data may follow an AR(3) model

The PACF is sinusoidal and there is a significant spike at lag 2 in the ACF So the data may follow an MA(2) model

So we propose three ARMA models for the differenced data: ARMA(p,q) ARMA(3,2), ARMA(3,0) and ARMA(0,2).

That is, for the original time series, we propose three ARIMA models,ARIMA(p,d,q) ARIMA(3,1,2), ARIMA(3,1,0) and ARMA(0,1,2).

Step 4: Build the ARIMA model

Manual ARIMA:

arima_fit_hosp_1 = Arima(daily_hosp_hb_timeseries, order = c(3,1,2))
arima_fit_hosp_2 = Arima(daily_hosp_hb_timeseries, order = c(3,1,0))
arima_fit_hosp_3 = Arima(daily_hosp_hb_timeseries, order = c(0,1,2))

summary(arima_fit_hosp_1)
summary(arima_fit_hosp_2)
summary(arima_fit_hosp_3)

texreg::screenreg(list(arima_fit_hosp_1, arima_fit_hosp_2, arima_fit_hosp_3),
                custom.model.names =c("ARIMA(3,1,2)","ARIMA(3,1,0)","ARIMA(0,1,4)"),
                center = TRUE,
                table = FALSE)

#Function for Automated ARIMA


auto_arima_fit_hosp <- auto.arima(lag(daily_hosp_hb_timeseries,k =lag_value ),
                  seasonal=FALSE,
                  stepwise=FALSE,
                  approximation=FALSE,
                  trace = TRUE
                  )

Automated ARIMA

#Lag is used to best fit the model
auto_arima_fit_hosp <- auto.arima(lag(daily_hosp_hb_timeseries),
                  seasonal=FALSE,
                  stepwise=FALSE,
                  approximation=FALSE,
                  trace = TRUE
                  )
auto_arima_fit_hosp

Automated ARIMA confirms that the ARIMA(3,1,2) seems good based on AIC

coef<-lmtest::coeftest(auto_arima_fit_hosp)
coef

All coefficients are significant except ar3.

Model Selection Criteria :

ARIMA models with minimum AIC, RMSE and MAPE criteria were chosen as the best models. Based on Akaike Information Criterion (AIC) above, an ARIMA(3, 1, 2) model seems best.

Step 5: Check for Diagnostics

Let’s plot the diagnostics with the results to make sure the normality and correlation assumptions for the model hold. If the residuals look like white noise, proceed with forecast and prediction, otherwise repeat the model building.

res <-checkresiduals(auto_arima_fit_hosp, theme = color_theme())
res

The ACF plot of the residuals from the ARIMA(3,1,2) model shows that all auto correlations are almost within the threshold limits, with residuals. A portmanteau test (Ljung-Box test) returns a smaller p-value , also suggesting that the residuals are white noise.

Fitting the ARIMA model with the existing data

The residual errors seem fine with near zero mean and uniform variance. Let’s plot the actuals against the fitted values

Convert model and time series to dataframe for plotting

daily_hosp_hb_timeseries_data <- fortify(daily_hosp_hb_timeseries) %>% 
  clean_names() %>% 
  remove_rownames %>% 
  rename (date = index,
          hosp = data)%>% 
  mutate(index = seq(1:nrow(daily_hosp_hb_timeseries)))
  
arima_fit_resid <- ts(daily_hosp_hb_timeseries[1:nrow(daily_hosp_hb_timeseries)]) - resid(auto_arima_fit_hosp)

arima_fit_data <- fortify(arima_fit_resid) %>% 
  clean_names() %>% 
  mutate(data = round(data,2))

fit_existing_data <- daily_hosp_hb_timeseries_data %>% 
  inner_join(arima_fit_data, by = c("index"))

Plotting the series along with the fitted values

fit_existing_hosp_plot <- fit_existing_data %>% 
  mutate (date = as.Date(date)) %>% 
  ggplot()+
  aes(x=date, y = hosp)+
  geom_line(color ="#5ab4ac")+
  geom_line(aes(x= date, y = data), colour = "red" )+
  xlab("Month") + 
  ylab("Patient Hospitalised")+
  theme(axis.text.x = element_text(angle = 45, vjust = 1, hjust=1))+
  ggtitle("Fitting the ARIMA model with existing data") +
  scale_x_date(breaks = "1 month", date_labels = "%b - %y" )+
  #scale_y_continuous(labels = scales::unit_format(unit = "M", scale = 1e-6))+
  color_theme()

ggplotly(fit_existing_hosp_plot)

Step 6 Forecast using the model

Data Preparation :

forecast_model <- forecast(auto_arima_fit_hosp,level = c(80, 95), h = 30) 

#Convert the model to dataframe for plotting

forecast_model_data <- fortify(forecast_model) %>% 
  clean_names() %>% 
  mutate(data = round(data,2),
         fitted= round(fitted,2)) %>% 
  mutate (lo_80 = ifelse(lo_80 < 0,0,lo_80),
          lo_95 = ifelse(lo_95 < 0,0,lo_95)
  )

forecast_start_date <- as.Date(max(daily_hosp_hb_timeseries_data$date)+1)
forecast_end_date <- as.Date(forecast_start_date+29)

forecast_data <- forecast_model_data %>% 
  filter(!(is.na(point_forecast))) %>% 
  mutate(date = seq(forecast_start_date,forecast_end_date, by =1)) %>% 
select(-data,-fitted, -index)  

fitted_data <- forecast_model_data %>% 
  filter(!(is.na(data))) %>% 
  inner_join(daily_hosp_hb_timeseries_data, by = c("index")) %>% 
  mutate(date = as.Date(date)) %>% 
select(date, data, fitted) 

Plotting the Vaccination series plus the forecast and 80 - 95% prediction intervals

annotation <- data.frame(
   x = c(as.Date("13-08-2021","%d-%m-%Y"),as.Date("31-10-2021","%d-%m-%Y")),
   y = c(180,200),
   label = c("PAST", "FUTURE")
)

#Time series plots for the next 60 days according to best ARIMA models with 80%–95% CI.
forecast_data_hosp_plot <-fitted_data %>% 
  ggplot()+
  geom_line(aes(x= date, y = data), color = "#5ab4ac")+
  #geom_line(aes(x= date, y = fitted), colour = "red" )+
  geom_line(aes(x= date, y =point_forecast), color ="blue", size = 0.5,
             data = forecast_data )+
  geom_ribbon(aes(x = date, y = point_forecast, ymin = lo_80, ymax = hi_80), 
              data = forecast_data, alpha = 0.3, fill = "green")+
  geom_ribbon(aes(x = date, y = point_forecast, ymin = lo_95, ymax = hi_95), 
              data = forecast_data, alpha = 0.1)+
  geom_vline(aes(xintercept=as.numeric(max(date))),color="#f1a340", linetype="dashed",data = fitted_data)+
  ggtitle("Projection of Hospitalisation") +
  xlab("Month") + 
  ylab("Patient Hospitalised")+
  theme(axis.text.x = element_text(angle = 45, vjust = 1, hjust=1))+
  color_theme()+
  scale_x_date(breaks = "1 month", date_labels = "%b - %y" )+
   geom_text(data=annotation, 
             aes( x=x, y=y, label=label),                  
            color="blue", 
            size=4 )
  
  ggplotly(forecast_data_hosp_plot)